The invention relates to systems and methods for computer imaging, and more particularly to systems and methods for locating the position of an image having a degraded or eroded portion.
Machine vision systems exist today that can allow for the automation of many manufacturing tasks, such as product assembly and inspection. To this end, machine vision systems typically include a camera that can be disposed above a workpiece, such as a computer board being assembled or an item being inspected. The camera can connect to a frame grabber board that can be incorporated into a computer workstation. Under the control of software operating on the computer, the camera can capture images of the workpiece and the frame grabber board can generate image frames, each of which can be stored as a computer file that can be processed by the software running on the computer workstation. The software operating on the computer workstation can analyze the image files to identify and locate within the image patterns that are known to the system. In this way, the machine vision system can determine the presence or absence of particular items within the image as well as the location of those items. This information can be provided to a manufacturing device, such as a solder tool, to allow the manufacturing device to perform its function such as by further assembling components to a computer board.
Although machine vision systems work quite well, challenges still remain. For example, the quality of images generated by a camera depends in part on conditions that can vary uncontrollably in the manufacturing environment. For example, the quality of an image generated by a camera turns in part on the quality of background lighting. However, in the manufacturing environment the background lighting can change constantly. This in turn can result in differences between image frames collected by the machine vision system. In particular, dark spots and reflective spots within the image can vary from frame to frame, thereby making it difficult to recognize or locate a known pattern within the image. For example, during a metal processing step, a solder joint can be polished to a highly reflective finish. Under some lighting conditions, the solder joint can appear as a solid brightly lit ball. However, under other conditions shadows can appear on the solder joint causing an image to form that looks like a partially clouded ball. For either image, the machine vision system is provided with the challenge of identifying the solder joint ball within the image. However, the differences between images can make it difficult for the machine vision system to recognize or locate the degraded solder joint image.
Additional problems arise in the manufacturing environment wherein debris, such as dust, can land on a workpiece being imaged and processed. Accordingly, portions of the items that are to be recognized by the machine vision system can be obscured or clouded, thereby making recognition difficult. Still other problems arise from changes in reflectance. For example, during a manufacturing process that sequentially deposits different materials onto a substrate surface, the reflectance of the surface can change significantly. These changes in reflectance can obscure portions of the surface, making recognition difficult.
In any case, the difficulty of recognizing degraded and decomposed images arises in part from the fact that the machine vision system is performing a pattern identification process wherein an image frame is searched to identify a known pattern within the image. The search process employs a pattern matching process wherein templates stored within the machine vision system are applied to portions of the image until a match between a template and a portion of the image can be found. A match is understood to represent the identification of a known item within the image. However, in those cases where the image includes degraded portions, the pattern matching process can fail as the image fails to contain an item that can confidently be matched to the correct template stored within the machine vision system. Additionally, even if the pattern can be found in the image, the exact location of the pattern may be difficult to determine, as the degraded image information may interfere with the exact measurement of the image position.
Accordingly there is a need in the art for improved systems for being able to identify and locate patterns in an image that includes degraded or decomposed portions.
The systems and methods described herein are directed to machine vision systems that provide location systems that can locate with sub-pixel accuracy objects or fiducial patterns in an image, even when the objects vary in size, orientation, shape, occlusion, and appearance. To this end, the machine vision systems described herein can employ an artificial intelligence based technique to locate a pattern within the image to a very high accuracy. In one practice, a technique is employed wherein an image is subdivided into a plurality of sub-images. Each of the sub-images is compared to a portion of a template image and a measure of the similarity between the sub-image and the portion of the template is generated. For example, in one process on a pixel-by-pixel basis the system compares the sub-image to a portion of the template. The system then determines which portion of the sub-image is most representative of a particular characteristic of the template, such as which portion of the sub-image is most likely to represent an outside edge of the object being located. Once the portion of the sub-image that is most likely to be representative of the outside edge is determined, a calculation can be performed to determine where the center of the object would be given the location of the edge and the sub-image. For each sub-image this process can be determined with the results being an array of measurements each being representative of a reference point or points, such as the center of the object, as determined by the portion of the object displayed in the sub-image. In a subsequent step, the systems sort through the values in the array to identify those sub-images that yielded values that deviated substantially from the normal value. These sub-images are deemed to include degraded or obstructed views of the object. Conversely, sub-images that yield similar values for the center of the object can be understood as containing trustworthy data. The system can then employ the trustworthy measures of the center of the object to determine more exactly the object center.
More specifically, in one aspect the invention provides processes for locating a pattern within an image that can comprise the acts of providing a template representative of the pattern to be located within an image. The image can be subdivided into a plurality of sub-images, each being representative of a portion of the image. The process can then compare each of the sub-images to the template to generate a plurality of score signals representative of a location, or candidate locations, of the pattern, and can then process these score signals to determine a final measure of location for the pattern. This final measure can be deemed a more accurate representation of the pattern""s location. In one practice, the step of processing the score signals can include a step of identifying at least one sub-image that includes a degraded image. As described above, a degraded image can include an image that has had a portion obscured by shadows or debris. Additionally, an image can be degraded at spots of high-reflection that create glare and wash-out portions of the image. Those of ordinary skill will know of other factors that can also degrade an image and these factors can also be addressed by the systems described herein.
To process the score signals, the system described herein, in one practice, apply an artificial intelligence grouping process that sorts the score signals into groups. One particular example of such a grouping process is a clustering process wherein the clustering process sorts the score signals into clusters and sorts the clusters to identify at least one cluster representative of sub-images having location information suitable for use in determining the location of the pattern. Other sorting and analytical techniques suitable for processing the score signals can be practiced with the invention without departing from the scope thereof.
Various implementations of the processes can be achieved, including those in which the step of comparing the sub-images to the template includes a step of processing the sub-images to generate a reference point signal that can be representative of the location of the center of the pattern. Optionally, the reference point signal can be determined to sub-pixel accuracy. For example, in one practice the center location is determined first to within one pixel location. Then the signal intensity values surrounding that identified pixel are fit to a surface, such as a parabola, to provide a sub-pixel level estimate of the center location. The step of subdividing the image can include a step of subdividing the image into a user-selected number of sub-images. Alternatively, an artificial intelligence process can be employed to determine a pattern of sub-images that can be generated from the image, wherein the pattern applied is understood to produce optimal or acceptable results for a subsequent pattern location process.
In a further aspect, the invention includes systems for locating a pattern within an image. These systems can comprise a data memory having storage for a template representative of the pattern; an image processor for subdividing the image into a plurality of sub-images, each being representative of a portion of the image; a comparator for comparing a plurality of the sub-images to the template to generate a plurality of score signals each being representative of a location for the pattern, and a locator for processing the score signals to determine a location for the pattern. The comparator can include a score filter mechanism for processing the score signals to identify any of the sub-images that include degraded image data. To this end, the systems can include a data processor that can apply an artificial intelligence grouping process to the score signals for sorting the score signals. One such processor can be a clustering processor for sorting the score signals into clusters and sorting the clusters to identify at least one cluster representative of sub-images having location information suitable for use in determining the location of the pattern.
The systems can also include an image processing means for processing a sub image to generate a center signal representative of a location of the center of the pattern. Additionally, the systems can include a sub-pixel location processor for processing signals, such as intensity signals, to generate a location signal representative of a location occurring between centers of two or more pixels in the image and representative of the location of the pattern. In one example, the system includes a processor for determining with sub-pixel accuracy the location signal. To subdivide the image, the system can provide a user interface for collecting criteria from a user that is representative of an instruction for subdividing the image into a plurality of sub-images. Alternatively, the system can include a subdividing mechanism for applying an artificial intelligence process to determine a pattern of sub-images to be generated from the image.
In a further aspect, the invention can be understood as a process or a system that determines a location of a pattern within an image signal that is comprised of a plurality of pixels. For example, such a system can comprise an image processor for subdividing the image signal into a plurality of sub-images each being representative of a portion of the image signal. A filter processor can be employed for processing the sub-images to filter out sub-images that fail to contain information representative of the location of the pattern within the image signal. A locator can be employed for processing sub-images that have information representative of the location of the pattern within the image. The locator can include a sub-pixel location processor capable of processing a plurality of signals representative of likely locations of the pattern. This allows the system to generate a location signal representative of a location that occurs between two or more pixels in the image and that is representative of the location of the pattern, or that can be understood to be representative of the location of the item that corresponds to the pattern in the image.
Additionally, it will be understood that the invention can be realized as computer programs that can be embodied in computer readable medium having stored thereon instructions for controlling a data processing system to implement a process according to the invention. One such process can comprise the steps of accessing a template representative of a pattern to be identified within an image, subdividing a signal representative of the image into a plurality of sub-images, each being representative of a portion of the image, comparing each of the sub-images to the template to generate a plurality of score signals representative of a location for the pattern, and processing the score signals to determine a location of the pattern within the image.
Other embodiments and practices of the invention will be understood from a review of the exemplary systems and methods illustrated herein.